Towards human-like conversational search systems

Voice search is currently widely available on the majority of mobile devices via use of Virtual Personal Assistants. However, despite its general availability, the use of voice interaction remains sporadic and is limited to basic search tasks such as checking weather updates and looking up answers to factual queries. Present-day voice search systems struggle to use relevant contextual information to maintain conversational state, and lack conversational initiative needed to clarify user’s intent, which hampers their usability and prevents users from engaging in more complex interaction activities. This research investigates the potential of a hypothesised interactive information retrieval system with human-like conversational abilities. To this end, we propose a series of usability studies that involve a working prototype of a conversational system that uses real time speech synthesis. The proposed experiments seek to provide empirical evidence that enabling a voice search system with human-like conversational abilities can lead to increased likelihood of its adoption

Let’s Talk it through, anew: Promises and Pitfalls of Customisable Conversational Reflection Support

As modern lifestyles are becoming increasingly stressful and ever more hectic with multiple stimuli constantly competing for our attention, Affective Disorders (ADs) such as anxiety and depression are on the rise. Consequently, due to the burgeoning demand for counseling and therapeutic services, many people who suffer from ADs are struggling to timely access the professional support that they require. To address this problem, voice-enabled Conversational Agents (CAs) have been recently proposed as tools for supporting self-reflection and providing assistance in managing a range of ADs through synthetic voices. However, despite their therapeutic potential, CAs offer a very limited choice when it comes to selection and personalisation of synthetic voices used. The goal of this paper is two-fold: (1) it discusses the potential benefits that a CA’s voice customisation can bring to enhance user engagement and promote long term self-reflection, and (2) it offers reflection on the corresponding challenges associated to this approach

Conversational strategies : impact on search performance in a goal-oriented task

Conversational search relies on an interactive, natural language exchange between a user, who has an information need, and a search system, which elicits and reveals information. Prior research posits that due to the non-persistent nature of speech, conversational agents (CAs) should support users in their search task by: (1) actively suggesting query reformulations, and (2) providing summaries of the available options. Currently, however, the majority of CAs are passive (i.e. lack interaction initiative) and respond by providing lists of results – consequently putting more cognitive strain on users. To investigate the potential benefit of active search support and summarising search results, we performed a lab-based user study, where twenty-four participants undertook four goal-oriented search tasks (booking a flight). A 2x2 within subjects design was used where the CAs strategies varied with respect to elicitation (Passive vs Active) and revealment (Listing vs. Summarising). Results show that when the CA’s elicitation was Active, participant’s task performance improved significantly. Confirming speculations that Active elicitation can lead to improved outcomes for end-users. A similar trend, though to the lesser extent, was observed for revealment – where Summarising results led to better performance than Listing them. These findings are the beginning of, but also highlight the need for, research into design and evaluation of conversational strategies that active or pro-active CAs should employ to support better search performance

Inquisitive Mind : A conversational news companion

With an ever-increasing amount of information and ever-more hectic lifestyles, many people rely on news briefs to stay up to date. Consequently, the reliance on single-source media narratives can lead to a biased and narrow perception of the world. Conversational interfaces, as a medium for delivering news stories, can help to address this problem by encouraging users to explore information resources and news stories by formulating curiosity driven comments and questions. We propose Inquisitive Mind (IM) - a conversational companion that proactively points out different narratives of the story, refers users to source materials, and encourages deeper exploration of the topic. We argue that IM could foster curiosity, encourage critical thinking, and effectively lead to more conscious media consumption

Affective Behaviour Analysis of On-line User Interactions: Are On-line Support Groups more Therapeutic than Twitter?

The increase in the prevalence of mental health problems has coincided with a growing popularity of health related social networking sites. Regardless of their therapeutic potential, On-line Support Groups (OSGs) can also have negative effects on patients. In this work we propose a novel methodology to automatically verify the presence of therapeutic factors in social networking websites by using Natural Language Processing (NLP) techniques. The methodology is evaluated on On-line asynchronous multi-party conversations collected from an OSG and Twitter. The results of the analysis indicate that therapeutic factors occur more frequently in OSG conversations than in Twitter conversations. Moreover, the analysis of OSG conversations reveals that the users of that platform are supportive, and interactions are likely to lead to the improvement of their emotional state. We believe that our method provides a stepping stone towards automatic analysis of emotional states of users of online platforms. Possible applications of the method include provision of guidelines that highlight potential implications of using such platforms on users' mental health, and/or support in the analysis of their impact on specific individuals

Modelling Attention Levels with Ocular Responses in a Speech-in-Noise Recall Task

We applied state-space modelling technique to estimate the cognitive workload of a speech-in-noise (SIN) recall task, based on participants’ oculo-motor responses to speech signals. We estimated common latent attention levels in 15 time bins and observed temporal changes between pupillary dilations and saccade frequencies, given that the both conditions were independent. We also compared two speech type factors (natural vs. synthetic) and three levels of signal-to-noise (-1dB, -3dB, and -5dB) using the estimated parameter distribution. The comparison of experimental factors provided us with insights into differences in participants’ processing of spoken information during a SIN recall task

Combining oculo-motor indices to measure cognitive load of synthetic speech in noisy listening conditions

Gaze-based assistive technologies (ATs) that feature speech have the potential to improve the life of people with communication disorders. However, due to a limited understanding of how different speech types affect the cognitive load of users, an evaluation of ATs remains a challenge. Expanding on previous work, we combined temporal changes in pupil size and ocular movements (saccades and fixation differentials) to evaluate cognitive workload of two types of speech (natural and synthetic) mixed with noise, through a listening test. While observed pupil sizes were significantly larger at lower signal-to-noise levels, as participants listened and memorised speech stimuli; saccadic eye-movements were significantly more frequent for synthetic speech. In the synthetic condition, there was a strong negative correlation between pupil dilation and fixation differentials, indicating a higher strain on participants’ cognitive resources. These results suggest that combining oculo-motor indices can aid our understanding of the cognitive implications of different speech types

Improving conversational dynamics with reactive speech synthesis

The active exchange of ideas and/or information is a crucial feature of human-human conversation. Yet it is a skill that present-day ‘conversational’ interfaces are lacking, which effectively hampers the dynamics of interaction and makes it feel artificial. In this paper, we present a reactive speech synthesis system that can handle user’s interruptions. Initial results of evaluation of our interactive experiment indicate that participants prefer a reactive system to a non-reactive one. Based on participants’ feedback, we suggest potential applications for reactive speech synthesis systems (i.e. interactive tutor and adventure game) and propose further interactive user experiments to evaluate them. We anticipate that the reactive system can offer more engaging and dynamic interaction and improve user experience by making it feel more like a natural human-human conversation

Conversational agents trust calibration

Previous work identified trust as one of the key requirements for adoption and continued use of conversational agents (CAs). Given recent advances in natural language processing and deep learning, it is currently possible to execute simple goal-oriented tasks by using voice. As CAs start to provide a gateway for purchasing products and booking services online, the question of trust and its impact on users' reliance and agency becomes ever-more pertinent. This paper collates trust-related literature and proposes four design suggestions that are illustrated through example conversations. Our goal is to encourage discussion on ethical design practices to develop CAs that are capable of employing trust-calibration techniques that should, when relevant, reduce the user's trust in the agent. We hope that our reflections, based on the synthesis of insights from the fields of human-Agent interaction, explainable ai, and information retrieval, can serve as a reminder of the dangers of excessive trust in automation and contribute to more user-centred CA design